1. Field of the Invention
The present invention relates to a display monitoring system for detecting and tracking a changing area in a picked-up image, especially, a moving object such as an intruder in a monitor area.
2. Description of the Related Art
Recently, various types of safety method have been studied to prevent crimes and the like in buildings and offices. For example, a display monitoring system has been put into practice. This system is designed to automatically detect an intruder or an intruding object in a monitor area by using an image pickup apparatus such as a TV camera. In order to detect intruding objects, many of such display monitoring systems employ a method of detecting a moving object by obtaining the differences between a plurality of images consecutively picked up at short time intervals, or the difference between a current image and a background image without any intruder or the like.
The following problems, however, are posed in the method of detecting a moving object by only detecting changes on the basis of such differences.
(1) In addition to an intruding object, noise caused by environmental variations is detected. It is difficult to discriminate an intruding object from the noise. Noise components caused by environmental variations correspond to changes in brightness due to illumination variations or changes in brightness at a given place due to reflection by a water surface or the sway of trees, and cause large differences in portions other than an intruding object. In the conventional method, therefore, objects to be detected cannot be reliably and selectively detected by removing the noise components.
(2) Since one changing area does not necessarily appear in a difference image of one intruding object, processing (e.g., removal of the above-mentioned noise, association of a plurality of changing areas originating from one object, determination of the presence/absence, number, and traces of intruding objects) subsequent to change area extraction processing cannot be accurately performed.
(3) Since influences of noise or the like caused in an picking up operation by an image pickup unit are detected together with an intruder or the like, it is very difficult to clearly separate the intruder or the like from the noise. That is, it is very difficult to detect one intruder as one area in an image.
As a method of solving the above-described problems, a block association processing scheme (Published Unexamined Japanese Patent Application Nos. 2-59976, 2-59977, and 2-59978) has been proposed.
In the block association processing scheme, as shown in FIG. 1, when an intruder is separated into a plurality of changing areas, i.e., head, body, and leg areas, in a difference image, corresponding change amounts between the changing area extracted at two consecutive time points are obtained by calculating feature amounts such as area and shape features. Moving vectors between the corresponding changing areas are then calculated and it is determined that the changing areas which have similar moving vectors originate from the same object. This scheme is effective as long as one object is always separated into the same constituent elements. In many cases, however, a changing area extracted on the basis of a difference does not necessarily correspond to a specific portion of an object.
For example, the reasons are:
(1) Changes in brightness due to illumination variations differently occur at the respective portions of a target object.
(2) If a target object is a three-dimensional object and has a depth and an uneven surface pattern, the appearance (area, shape, and the like) of the object changes in different directions.
(3) If a target object is not a perfectly rigid object (e.g., a man wearing clothing), the shape of the object changes at each time point.
(4) If a portion of a background and a portion of a target object have the same color or luminance, the same color or luminance portion of the target object is not detected as a changing area while the target object passes through the same color or luminance portion of the background.
In the states wherein corresponding changing areas are not necessarily detected at each time point as described above, since corresponding changing areas may not be present, moving vectors may not be obtained. Therefore, the block association processing scheme may not properly function. If the block association processing scheme fails or does not properly function as described above, serious problems are posed when it is applied to a monitoring system (display monitoring system).
In order to solve the above-described problems, a tracking means capable of tracking a moving object is employed in some cases. In the scheme using such tracking means, since a moving object is tracked, association between areas need not be performed. However, the tracking means of this scheme is used to analyze, e.g., the movement of a human being in many cases. According to the tracking method of the tracking means, a portion to be tracked, e.g., a marker (target) attached to an object to be measured (e.g., an arm or leg of a man), must be determined in advance. Therefore, it is impossible to sequentially detect many unspecified intruders in a monitor area and track them.
As a method of tracking unspecified targets in a monitor area, the following method is also proposed. This method employs a change detecting means for detecting a changing area which changes due to the movement of a moving object such as an intruder, with a tracking means arranged for the changing area detected by the change detecting means. With this arrangement, the moving object presumed to be present in the changing area is tracked.
For example, the following tracking methods are employed by the tracking means described above:
(1) tracking changes in brightness (pixel luminance) in a tracking area;
(2) tracking a specific form in a tracking area; and
(3) determining a moving direction by calculating the correlation value between a tracking area and peripheral areas.
Since a given tracking means can track only a portion of a moving object, the following method is also proposed. In this method, after tracking processing is completed, tracking results are analyzed, and the tracking means are divided into groups in units of moving objects (i.e., the tracking means are associated in groups on the basis of the moving directions of the tracking means and their positional relationship). In addition, as a method of simultaneously detecting many unspecified moving objects in a monitor area, a method of using a plurality of tracking means is proposed.
According to the above-described method, even if some of the tracking means groups fail to track because of external disturbances such as noise and blocking by an object, compensation of such failures can be properly performed. If, however, moving objects cross each other in a complicated manner, it is very difficult to interpret tracking results. When objects which are moving in different directions are to be tracked by tracking means, even if they belong to the same moving object, e.g., the arms and legs of a man, these tracking means cannot be associated with each other in the same group. In such a case, therefore, it is impossible to detect the moving object.
As described above, in the conventional display monitoring system, since the influences of the phenomenon that changes in brightness occur in portions other than an intruding object as a target object cannot be eliminated, a detection error is inevitably caused. Therefore, when a plurality of changing areas originate from one object, it is difficult to identify the object.
Moreover, when moving objects such as intruders are to be tracked and detected by using a plurality of tracking means, if a plurality of objects, each of which moves in a complicated manner like a man, exist in a detection area, it is impossible to simultaneously detect the respective intruders.